Universal bounds for the Holevo quantity, coherent information 
and the Jensen-Shannon divergence 
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The Holevo quantity provides an upper bound for the mutual information between the sender 
of a classical message encoded in quantum carriers and the receiver. Applying the strong sub- 
additivity of entropy we prove that the Holevo quantity associated with an initial state and a given 
quantum operation represented in its Kraus form is not larger than the exchange entropy. This 
implies upper bounds for the coherent information and for the quantum Jensen-Shannon divergence. 
Restricting our attention to classical information we bound the transmission distance between any 
two probability distributions by the entropic distance, which is a concave function of the Hellinger 
distance. 



I. INTRODUCTION 



The goal of quantum information is to efficiently apply 
quantum resources to encode, manipulate, and transmit 
information. One of the key results about transmitting 
classical information by quantum means is the Holevo 
bound: it provides an upper bound for the accessible 
information or, stated differently, quantum distinguisha- 
bility between initial states cannot be increased by mea- 
suring them [l|. 

Assume that a source emits messages written in an al- 
phabet X and that these messages are sent to a receiver 
using a quantum device. Each letters is encoded in 
a quantum state, i.e., in a density matrix pi. The re- 
ceiver performs a measurement on the encoded messages 
and obtains classical data written in an alphabet Y, The 
probability that the source emits a, is given by and so 
with high probability messages of length n will belong to 
a set containing about exprnH (X)) words where H is the 
Shannon entropy: H(X) :— — J^iftlngi. By performing 
a general POVM the receiver reads the letter bi with a 
probability pi . The accessible information is then the mu- 
tual information H(X : Y) = H(X) + H(Y) - H(X, Y), 
where H(X, Y) is the Shannon entropy of the joint prob- 
ability distribution. The fundamental result of Holevo 
gives [1, H| an upper bound for the mutual information, 
independent of the measurement: 



H(X : Y) < s(j2<HPi) -^QiSipi). 



(1) 



Here S(p) := —Trphip is the von Neumann entropy of 
the state p. The right-hand side of inequality (Q} 

x({lhPi}) ■= sCy^qipij -^2qiS(pi) (2) 



is the Holevo quantity of the ensemble {qi, p{\. 

It is well-known [l| that the Shannon entropy of the 
probability vector {qi} gives an upper bound for \; 



xiiliipi}) < I n this work we provide a better 

upper bound for the Holevo quantity and explore some 
of its consequences for classical and quantum information 
theory. 



II. QUANTUM OPERATIONS 



Consider a quantum system A in the state p interacting 
with an environment B initially in a pure state \<fi) € T-Lb- 
Any quantum operation $ on the system can by seen as 
a global unitary dynamics followed by the partial trace 
over the environment: 



$ : p H> p' = Tij 



,(l/(p0|$M)tft). (3) 



Here U is a unitary matrix of the total system AB and 
Ttb denotes the partial trace over the environment. The 
map $ is completely positive and can be represented in 
Kraus form: 



®{p)=Y,K iP Kl 



(4) 



where the operators Ki are determined by minors of U. 
Due to the unitarity of U the set of Kraus operators is 
a resolution of the identity: J^i K\Ki = t which implies 
that $ preserves the trace. 

We consider also a quantum map $ complementary to $ 
defined by the partial trace over the principal system 0, 



(5) 



The state a = $(p) is the state of the environment after 
the interaction and is called a correlation matrix. Its 
matrix elements can be expressed in terms of the Kraus 
operators: 



<7y = Tx pKjKi 



(6) 
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If the initial state p is pure then S(a) is the entropy 
exchanged between the system and the environment. 
Therefore S(a) is called the exchange entropy. 
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FIG. 1: a) A dynamical picture: an initial quantum state p 
is sent by a map $ into p' , while the complementary map <3> 
sends it into a. A Kraus operator Ki maps p into p'i with 
probability qi so that p' is the barycentre of the ensemble 
{luPi}- 

b) A static picture: the ensemble {qi, pi} determines the 
barycentre p. 

Due to the identity resolution the set of Kraus operators 
describes a Positive Operator- Valued Measure. Such a se- 
lective measurement transforms an initial state p into one 
of the output states p\ := KipKj /(Tr pK\Ki) with prob- 
ability qi — Tr pK\Ki. For this setup, shown in FigOJi, 
one defines the Holevo quantity x{{Qi> Pi}) > see ©• 



Using S(b}4,) = and SV24) = S(ui 13 ) 



we recover 
□ 



It is well-known that, in contrast to classical entropy, 
quantum entropy does not necessarily increase in the 
number of parties: S(u>i) ^ S(u>i2). Proposition [1] is 
therefore trivial in the classical case but bounds the sum 
of entropies of single party quantum states by that of en- 
tropies of joint extensions. Our main result is contained 
in the following bounds: 

Proposition 2. Consider a state p, a quantum operation 
(f> and the image of p under p' = &(p) = 53« KipK\- 
The complementary channel produces a correlation ma- 
trix a = <&(p) with elements specified in @). Define the 
probability vector with entries qi := TrpKjKi and quan- 
tum states p\ := KipKj/qi so that p' = qip'i- Then 

a) the Holevo quantity is bounded by the exchange en- 
tropy: 



< S(a) < H({ qi }) and 



(11) 



b) the average entropy is bounded by the entropy of the 
initial state, 



(12) 



III. THE MAIN RESULT 



To prove our main result we first reformulate the strong 
sub-additivity inequality for quantum entropy. 

Proposition 1. Let u)i 23 be a three party quantum state, 
then 



S(u>i) + S(uj 3 ) < S(uj 12 ) + S(u 23 ). 



(7) 



Conversely, if ^ holds for any three party quantum state 
then also 



SV123) + S(lu 2 ) < S(u>i 2 ) + S(uj 23 ). 



(8) 



In these inequalities reduced states are obtained by trac- 
ing out over the complementary subsystems, e.g., u)% = 
Tr 23 W123. 

Proof. The proof relies on purification. Indeed, purifying 
W123 to wi 23 4 we have the strong sub-additivity inequality 



SV234) + S(UJ 3 ) < S{ll>23) + S{iL> 34 ). 



(9) 



As wi 23 4 is pure, S(u} 23 4) = S'(wi) and 5(o; 34 ) = S(uji 2 ) 
which turns ([9]) into (J7]). 

Conversely, if ([7]) holds, we apply it to a purification W1234 
of u\ 23 to obtain 



S(oj 2 ) + S(u^) < S(u} 23 ) + S(u} 2 i)- 



(10) 



Proof, a) The rightmost inequality: S(a) < ff({g,}) is 
a direct consequence of the majorization theorem which 
says that for any state S(a) < S(diag(cr)), see e.g. @. To 
prove the left inequality consider the isometry F\<p) :— 
J2i N) ® \i) €5 Ki\4>) an( l the three-partite quantum state 

W123 := Fpft = ® |i>0'| ® KipKt. (13) 



It is convenient to introduce the notation Ay :— KipK-, 
so that qi = Tr An and p[ = An / qi . One checks that 

SV12) = S(a) 

S{oj 3 ) = S(^2 qi pt) and 

i 

i i i 

= S(ui) - S(lj 23 ). (14) 

Substituting these expressions in ([7]) yields the first in- 
equality in (fTTj) . 

b) Since the transformation F in (fTB")) is an isometry the 
three-partite state W123 has the same spectrum as p up to 
multiplicities of zero. Hence, W123 and p have the same 
entropy. The equality (TBI and the Araki-Lieb inequality 
SVi) - SV23) < SV123) then yield (12). ~ □ 
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The bounds in proposition [2] are universal, they hold 
for any quantum operation <f> and any initial state p. 
We analyze here some of their consequences. The in- 
equality (fTT|) is saturated for orthogonal Kraus opera- 
tors, Trie/ K j = SijKi = SijKj which form a projec- 
tive von Neumann measurement. In this case all output 
states p\ are pure, so J^iQi^iPi) — 0- The state p' is 
a mixture of pure and orthogonal states with probabil- 
ities qi = Tr pK[ Ki . The correlation matrix a is then 
diagonal and S(p') = S(a). 

Note also that inequality (fl2|) differs from S = 
J2i QiS(Pi) — S(p')> which is implied by the concavity of 
entropy. For a bistochastic map $ the entropy does not 
decrease, so in this case we may write S < S(p) < S(p'). 

The Jamiolkowski isomorphism represents of a quantum 
map $ acting on an TV-level system by a density matrix 
er$ on an extended space: a<s> — $ <g> id(|0 + )(0 + |), where 
10+) = -j= \i) ® \i) is a maximally entangled state. 
The degree of non-unitarity of an operation <f> can be 
quantified by its entropy @ , defined as the entropy of the 
corresponding state £($) := 5(<7$). If the initial state 
p is maximally mixed, then the exchange entropy S(a) 
is equal to the entropy of the map [3] ■ Proposition [5] 
yields now a simple interpretation of the entropy of a 
map: it is an upper bound for the Holevo quantity ([2]) for 
a transformation of the maximally mixed state = 1/iV. 
Furthermore, the entropy of a map is an upper bound for 
the Holevo quantity associated with ensembles of Kraus 
maps acting on the mixed state p* 

max X ({Tr KiKj/N, < (15) 

{Ki} VI TrJQirtJ/ 

where the maximum is taken over sets of Kraus opera- 
tors that realize the same quantum operation: <&(p) = 



p\ := KipKj/qi. Then 

a) The coherent information for the quantum operation 
$1 is bounded by 

Ioohi^i) <YlliS(p'd < S(p). (17) 

i 

b) The coherent information for the concatenation $2°$i 
is bounded by 

/ coft ($ 2 °$i)<5>iS($ 2 (X)). (18) 

i 

Proof. Relation (TT71) is a direct consequence of propo- 
sition [2J as these inequalities are obtained by combin- 
ing (fTTj) and (H2). To show (|18p we consider the four- 
partite state 

^234 := E l*> 01 ® l fc >W ® ®LiK kP K\L], (19) 

where ^(p) = J2i^iP^l- Consider the strong sub- 
additivity relation 

S{uQ + S(u/ 13 ) < S{u/ in ) + S{J 2i ). (20) 

The left-hand side inequality (|18|) which we want to prove 
can be rewritten as 

S(^)+S(lj 3 )<S(lj' 123 )+S(cj 24 ). (21) 

Therefore, it is sufficient to prove that S(cj' 3 ) < S(ui' 13 ). 
As the matrix ui' 3 is diagonal and consists of the traces of 
the blocks of the block diagonal matrix uj[ 3 , uj' 13 is more 
mixed than oj' 3 and has therefore larger entropy. □ 



V. THE JENSEN-SHANNON DIVERGENCE 



IV. COHERENT INFORMATION 

Investigating the entropy transfer induced by a quantum 
map <E> that sends a state p to p' = $(p) = Yli^iP^I^ 
Lindblad 8] proved the inequality 

S(p')-S(a)<S(p)<S(p')+S(a). (16) 

The correlation matrix a is defined in ([B]) and the proof 
is based on sub-additivity of entropy and on the Araki- 
Lieb triangle inequality. The difference of entropies, 
/ C oh := S($(p)) — S(a) is called coherent information. 
Linblad's inequality states that i co h < S(p). We are now 
in position to refine this bound. 

Proposition 3. Consider a state p and quantum op- 
erations $i and $2, with $i(p) — J2i^-iP^i an d a 
quantum ensemble {qi,p'i\ where qi :— Tr pKjKi and 



Let us now consider the static case in Fig. lb: a quan- 
tum ensemble {qi, Pi} which determines the average state 
p := YliliPi- For an ensemble of classical measures, 
{qi,Hi} one defines the generalized Jensen- Shannon di- 
vergence (JSD) by 

J5({ % , M i}) :=Jj(2ffi/ii) -Y^qiHQii), (22) 

i i 

which is an exact classical analogue of the expression © . 
Hence the Holevo quantity x is often called the Quantum 
Jensen-Shannon divergence (QJSD) [9l4ll|. 

It is intuitively clear that the results in Section IIIII may 
be used to derive upper bounds for QJSD, although the 
map $ in Proposition [2] and its Kraus form are not 
specified here. For any initial state p and an arbitrary 
Kraus operator Ki we consider the polar decomposition 
Kip 1 / 2 = XiUi. Here X, is a Hermitian matrix and J7, 
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is unitary. Note that Xf = KipKj, and this is equal to 
qip'i- Therefore Ki^fp — y/qip'JJi, so the elements of the 
correlation matrix © read: 



ay =Ti K iP K]K i = 
In this way we arrive at 



WiU}VPj- (23) 



Corollary 4. Consider a quantum ensemble {qi,Pi} and 
a collection of unitary matrices {Ui} and construct the 
correlation matrix a as in i23\) . Then x({<7i)Pi}) ^ ^(a). 

As a simple application consider the case of an ensemble 
with 2 elements. To obtain the lowest upper bound for 
Q JSD we need to minimize the entropy of the correlation 
matrix (|23|) over sets of unitaries {U\, U^}- This is equiv- 
alent to finding the POVM which minimizes S(a) among 
all measurements which result in the same ensemble of 
output states. 

Lemma 5. Consider two density matrices p\ and pi oc- 
curring with probabilities (A, 1 — A). The smallest entropy 
of the correlation matrix i23\) over unitaries U\ and U2 
is achieved for the matrix 



A _ y/\(l-\)VF 

v/A(l - X)\/F 1 - A 



(24) 



where y/F is the root fidelity fFH /: y/F = Tr y pi p2P\ ■ 

Proof. Given A, p±, and P2 the entropy S(o~) is minimal, 
if the absolute value of the off-diagonal element <r 12 is 
maximal. As |TrA_B| < ||a|j Tr|-B| we have the upper 
bound 



Tr pi pi UiUl\ < Tr | = VF 



(25) 



Moreover the inequality is saturated by choosing for 

U\U\ the adjoint of the unitary of the polar decompo- 

11 

sition of pi pi ■ □ 



Let us now set A = 1 — A = |. In this case the 
quantum Jensen-Shannon divergence can be written as 
QJS(pi,p 2 ) = ± [S{ Pl \\p) + S(p 2 \\p)], where p = \ {pi + 
P2) and S(pi\\p 2 )-= Trpi(lnpi — ln/52) is the quantum 
relative entropy [l| . The spectrum of the correlation ma- 
trix ([24]) is (/i, 1 — p) with pj = I (1— \/F). Hence we get 
an explicit formula for the exchange entropy S(a) and 
the universal bound for QJSD 



Q3S(px,p 2 ) < H 2 (\ (1 - y/F{ Pl ,p 2 )) 



(26) 



where H 2 {x) :— — xlnx— (1— a;) In (1 — x) is the Shannon 
entropy of a probability vector of size 2. 

The right-hand side of (l2"rJ|) can be used to character- 
ize closeness between quantum states. Although the en- 
tropy H2 does not obey the triangle inequality its square 



root does. Such an entropic distance was advocated by 
Lamberti et al. [l3[ as a natural metric in the space of 
quantum states: 



D, 



KPl,P2) := \jH2(j(l-y/F( Pl ,p 2 ))). 



(27) 



To show that D% is a distance one may use the Bu- 

res distance, Db{pi,P2) = y^2 — 2y/F(pi, p 2 ). Since 
both quantities are functions of fidelity, one can write 
the entropic distance as a function of the Bures distance 
De(Db) = \jH 2 {D B / / i). As the second derivative of 
De(Db) is negative, this function is concave, so De sat- 
isfies the axioms of a distance. 

Turning now to the classical case, i.e., diagonal density 
matrices, the root fidelity reduces to the Bhattacharyya 
coefficient B(P, Q) = J^i y/Wli while the Bures distance 
is equivalent to the Hellinger distance @ Dh(P,Q) — 
\Z^l2i(\/Pi ~ V^i) 2 - The entropic distance De between 
two classical states is then a con cave function of their 
Hellinger distance, D E (P,Q) = y/H 2 {D 2 H {P 7 Q)/4). Al- 
though JSD does not satisfy the triangle inequality, its 
square root does [lH and is called the transmission dis- 
tance [ll|. Inequality (|26|) implies thus the following re- 
lation between the transmission distance Dt and the en- 
tropic distance De used in [liij . 



D T {P,Q) := y/D3S(P,Q) < D E (P,Q), (28) 
which is illustrated in Fig. O 




FIG. 2: The transmission distance Dt (solid line) is bounded 
from above by the entropic distance De (dashed line) 

a) for pairs of classical states P — (p,l—p) and Q = (1 — p, p) 

b) for pairs of classical states P = (p, 1 — p) and Q = (1,0). 



VI. CONCLUSION 



In this work we showed that the Holevo quantity \ is 
bounded by the exchange information S(cr) and we ana- 
lyzed some consequences of this result. In particular, we 
showed that the transmission distance Dt between clas- 
sical states is bounded by their entropic distance De- 
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